{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div align=\"right\" style=\"text-align: right\"><i>Peter Norvig, Oct 2017<br>pandas Aug 2020<br>Data updated monthly</i></div>\n",
    "\n",
    "# Bike Stats Code\n",
    "\n",
    "Code to support the analysis in the notebook [Bike-Stats.ipynb](Bike-Stats.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import HTML\n",
    "from typing import Iterator, Iterable, Tuple, List, Dict\n",
    "from collections import namedtuple\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy  as np\n",
    "import pandas as pd\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reading Data: `rides`, `yearly`, and `daily`\n",
    "\n",
    "I saved a bunch of my recorded [Strava](https://www.strava.com/athletes/575579) rides, most of them longer than 25 miles, as [`bikerides.tsv`](bikerides.tsv).  The tab-separated columns are: the date; the year; a title; the elapsed time of the ride; the length of the ride in miles; and the total climbing in feet, e.g.: \n",
    "\n",
    "    Mon, 10/5/2020\tHalf way around the bay on bay trail\t6:26:35\t80.05\t541\n",
    "    \n",
    "I parse the file into the pandas dataframe `rides`, adding derived columns for miles per hour, vertical meters climbed per hour (VAM), grade in feet per mile, grade in percent, and kilometers ridden:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def parse_rides(lines):\n",
    "    \"\"\"Parse a bikerides.tsv file.\"\"\"\n",
    "    return drop_index(add_ride_columns(pd.read_table(lines, comment='#',\n",
    "                       converters=dict(hours=parse_hours, feet=parse_int))))\n",
    "\n",
    "def parse_hours(time: str) -> float: \n",
    "    \"\"\"Parse '4:30:00' => 4.5 hours.\"\"\"\n",
    "    hrs = sum(int(x) * 60 ** (i - 2) \n",
    "              for i, x in enumerate(reversed(time.split(':'))))\n",
    "    return round(hrs, 2)\n",
    "\n",
    "def parse_int(field: str) -> int: return int(field.replace(',', '').replace('ft', '').replace('mi', ''))\n",
    "\n",
    "def add_ride_columns(rides) -> pd.DataFrame:\n",
    "    \"\"\"Compute new columns from existing ones.\"\"\"\n",
    "    mi, hr, ft = rides['miles'], rides['hours'], rides['feet']\n",
    "    if 'date' in rides and 'year' not in rides:\n",
    "        rides.insert(1, \"year\", [int(str(d).split('/')[-1]) for d in rides['date'].tolist()])\n",
    "    return rides.assign(\n",
    "        mph=round(mi / hr, 2),\n",
    "        vam=round(ft / hr / 3.28084),\n",
    "        fpmi=round(ft / mi),\n",
    "        pct=round(ft / mi * 100 / 5280, 2),\n",
    "        kms=round(mi * 1.609, 2),\n",
    "        meters=round(ft * 0.3048))\n",
    "\n",
    "def drop_index(frame) -> pd.DataFrame:\n",
    "    \"\"\"Drop the index column.\"\"\"\n",
    "    frame.index = [''] * len(frame)\n",
    "    return frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "rides  = parse_rides(open('bikerides.tsv'))\n",
    "\n",
    "yearly = parse_rides(open('bikeyears.tsv')).drop(columns='date')\n",
    "\n",
    "daily  = yearly.copy()\n",
    "for name in 'hours miles feet kms meters'.split():\n",
    "    daily[name] = round(daily[name].map(lambda x: x / (6 * 50)), 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reading Data: `segments` and `tiles`\n",
    "\n",
    "I picked some representative climbing segments ([`bikesegments.csv`](bikesegments.csv)) with the segment length in miles and climb in feet, along with several of my times on the segment. A line like\n",
    "\n",
    "    Old La Honda, 2.98, 1255, 28:49, 34:03, 36:44\n",
    "    \n",
    "means that this segment of Old La Honda Rd is 2.98 miles long, 1255 feet of climbing, and I've selected three times for my rides on that segment: the fastest, middle, and slowest of the times  that Strava shows. (However, I ended up dropping the slowest time in the charts to make them less busy.)\n",
    "\n",
    "I keep track of percentage of roads ridden in various places in `'bikeplaceshort.csv'`, which comes from wandrer.earth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def parse_segments(lines) -> pd.DataFrame:\n",
    "    \"\"\"Parse segments into rides. Each ride is a tuple of:\n",
    "    (segment_title, time,  miles, feet_climb).\"\"\"\n",
    "    records = []\n",
    "    for segment in lines:\n",
    "        title, mi, ft, *times = segment.split(',')[:5]\n",
    "        for time in times:\n",
    "            records.append((title, parse_hours(time), float(mi), parse_int(ft)))\n",
    "    return add_ride_columns(pd.DataFrame(records, columns=('title', 'hours', 'miles', 'feet')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_clickable(comment) -> str:\n",
    "    \"\"\"Make a clickable link for a pandas dataframe.\"\"\"\n",
    "    if '!' not in comment:\n",
    "        return comment\n",
    "    anchor, number = comment.split('!')\n",
    "    return f'<a href=\"https://www.strava.com/activities/{number}\" rel=\"noopener noreferrer\" target=\"_blank\">{anchor}</a>'\n",
    "\n",
    "def link_date(date) -> str:\n",
    "    \"\"\"Make the date into a clickable link.\"\"\"\n",
    "    m, d, y = date.split('/')\n",
    "    return f'<a href=\"https://www.statshunters.com/?to={y}{m}{d}\" rel=\"noopener noreferrer\" target=\"_blank\">{date}</a>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "segments = parse_segments(open('bikesegments.csv'))\n",
    "\n",
    "tiles = drop_index(pd.DataFrame(columns='date square cluster total comment'.split(), data=[\n",
    "    ('01/01/2025', 14, 1395, 3520, 'Start of 2025'),\n",
    "    ('09/21/2024', 14, 1394, 3496, 'Michael J. Fox ride in Sonoma!12470434052'),\n",
    "    ('04/28/2024', 14, 1275, 3382, 'Livermore!11287081291'),\n",
    "    ('02/25/2024', 14, 1196, 3279, 'Expanding through Santa Cruz and to the South!10838162005'),\n",
    "    ('01/01/2024', 14, 1056, 3105, 'Start of 2024'),\n",
    "    ('12/08/2023', 14, 1042, 3084, 'Benicia ride connects East Bay and Napa clusters!10350071201'),\n",
    "    ('11/05/2023', 14,  932, 2914, 'Alum Rock ride gets 14x14 max square!8850905872'),\n",
    "    ('06/30/2023', 13,  689, 2640, 'Rides in east Bay fill in holes!9298603815'),\n",
    "    ('04/14/2023', 13,  630, 2595, 'Black Sands Beach low-tide hike connects Marin to max cluster!8891171008'),\n",
    "    ('03/04/2023', 13,  583, 2574, 'Almaden rides connects Gilroy to max cluster!8654437264'),\n",
    "    ('10/22/2022', 13,  396, 2495, 'Alviso levees to get to 13x13 max square!8003921626'),\n",
    "    ('10/16/2022', 12,  393, 2492, 'Milpitas ride connects East Bay to max cluster!7974994605'),\n",
    "    ('09/08/2022', 11,  300, 2487, 'First started tracking tiles')])\n",
    "    ).style.format({'comment': make_clickable, 'date': link_date})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plotting and Curve-Fitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams[\"figure.figsize\"] = (12, 6)\n",
    "\n",
    "def show(X, Y, data, title='', degrees=(2, 3)): \n",
    "    \"\"\"Plot X versus Y and a best fit curve to it, with some bells and whistles.\"\"\"\n",
    "    grid(); plt.ylabel(Y); plt.xlabel(X); plt.title(title)\n",
    "    plt.scatter(X, Y, data=data, c='grey', marker='+')\n",
    "    X1 = np.linspace(min(data[X]), max(data[X]), 100)\n",
    "    for degree in degrees:\n",
    "        F = poly_fit(data[X], data[Y], degree)\n",
    "        plt.plot(X1, [F(x) for x in X1], '-')\n",
    "    \n",
    "def grid(axis='both'): \n",
    "    \"Turn on the grid.\"\n",
    "    plt.minorticks_on() \n",
    "    plt.grid(which='major', ls='-', alpha=3/4, axis=axis)\n",
    "    plt.grid(which='minor', ls=':', alpha=1/2, axis=axis)\n",
    "    \n",
    "def poly_fit(X, Y, degree: int) -> callable:\n",
    "    \"\"\"The polynomial function that best fits the X,Y vectors.\"\"\"\n",
    "    coeffs = np.polyfit(X, Y, degree)[::-1]\n",
    "    return lambda x: sum(c * x ** i for i, c in enumerate(coeffs)) \n",
    "\n",
    "estimator = poly_fit(rides['feet'] / rides['miles'], \n",
    "                   rides['miles'] / rides['hours'], 2)\n",
    "\n",
    "def estimate(miles, feet, estimator=estimator) -> float:\n",
    "    \"\"\"Given a ride distance in miles and total climb in feet, estimate time in minutes.\"\"\"\n",
    "    return round(60 * miles / estimator(feet / miles))\n",
    "\n",
    "def top(frame, field, n=20): return drop_index(frame.sort_values(field, ascending=False).head(n))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Wandrer Places "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mapl(f, *values): return list(map(f, *values))\n",
    "\n",
    "places   = drop_index(pd.read_table(open('bikeplaceshort.csv'), sep=',', comment='#'))\n",
    "\n",
    "def wandrer(places=places, by=['pct', 'name'], ascending=[False, True], county=None):\n",
    "    \"All those who wander are not lost.\" # Also try by=['county', 'pct']\n",
    "    if county:\n",
    "        places = places[places.county == county]\n",
    "    F = drop_index(places.sort_values(by=by, ascending=ascending))\n",
    "    pd.set_option('display.max_rows', None)\n",
    "    return pd.DataFrame(\n",
    "        {'name':   F['name'],\n",
    "         'county': F['county'],\n",
    "         'total':  F['miles'],\n",
    "         'done':   [rounded(m * p / 100) for m, p in zip(F['miles'], F['pct'])],\n",
    "         'pct':    [pretty_pct(p) for p in F['pct']], \n",
    "         'badge':  [badge(float(p)) for p in F['pct']],\n",
    "         'to next badge': [to_go(p, m) for p, m in zip(F['pct'], F['miles'])],\n",
    "         'to big badge': [to_go(p, m, {25: .25, 90: .50}) for p, m in zip(F['pct'], F['miles'])]\n",
    "        })\n",
    "\n",
    "def pretty_pct(pct) -> str:\n",
    "    return '100%' if pct == 100 else f'{pct:.2f}%' if pct > 1 else f'{pct:.4f}%'\n",
    "\n",
    "def badge(pct) -> str:\n",
    "    \"\"\"What badge has this got us?\"\"\"\n",
    "    for badge in (99, 90, 75, 50, 25):\n",
    "        if pct >= badge:\n",
    "            return f'{badge}%'\n",
    "    return 'none'\n",
    "\n",
    "bonuses = {0.02: 0, 0.1: 0, 0.2: 0, 1: 0, 2: 0, 25: .25, 50: .05, 75: .10, 90: .50, 99: .10}\n",
    "\n",
    "def to_go(pct, miles, bonuses=bonuses) -> str:\n",
    "    \"\"\"Describe next target to hit to get a badge.\"\"\"\n",
    "    done = pct * miles / 100\n",
    "    for b in bonuses:\n",
    "        if done < b / 100 * miles:\n",
    "            delta = b / 100 * miles - done\n",
    "            return f'{rounded(delta):>5} mi to {b}% ({rounded(bonuses[b] * miles + delta)} points)'\n",
    "    return ''\n",
    "        \n",
    "def rounded(x: float) -> str: \n",
    "    \"\"\"Round x to 3 spaces wide (if possible).\"\"\"\n",
    "    return (rounded(x/1e6) + 'M' if x > 1e6\n",
    "            else f'{x/1e6:4.2f}M' if x > 1e5\n",
    "            else f'{round(x):,d}' if x > 10 \n",
    "            else f'{x:.1f}')\n",
    "\n",
    "other_places = places[~places.county.isin(['---', 'SMC', 'SCC', 'SFC', 'ALA'])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SMC / SCC Leaders "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_leaders(raw_data):\n",
    "    \"\"\"Make a dataframe of leaders in two counties.\"\"\"\n",
    "    data = [(name, SMp, SCp, *county_miles(SMp, SCp), round((SMp * SCp) ** 0.5, 2), round((SMp + SCp) / 2, 2), initials(name))\n",
    "            for (name, SMp, SCp) in raw_data]\n",
    "    leaders = pd.DataFrame(data, columns=[\n",
    "        'Name', 'SMC %', 'SCC %', 'SMC miles', 'SCC miles', 'Total miles', 'GeoMean %', 'Mean %', 'Initials'])\n",
    "    return drop_index(leaders)\n",
    "\n",
    "def county_miles(SMp, SCp) -> list:\n",
    "    SMmiles = round(2827.3 * SMp / 100)\n",
    "    SCmiles = round(7688.7 * SCp / 100)\n",
    "    return [SMmiles, SCmiles, SMmiles + SCmiles] \n",
    "\n",
    "def initials(name: str) -> str:\n",
    "    \"\"\"First and last initials.\"\"\"\n",
    "    return name[0] + name.split()[-1][0]\n",
    "\n",
    "def plot_leaders(leaders, by='Mean %'):\n",
    "    leaders = leaders.sort_values(by=by, ascending=False)\n",
    "    ax = leaders.plot('SMC %', 'SCC %',  kind='scatter', marker='D')\n",
    "    front = sorted((x, y) for i, (_, _, x, y, *_) in leaders.iterrows())\n",
    "    \"ax.axis('square')\"; grid()\n",
    "    ax.set_xlabel('San Mateo County %')\n",
    "    ax.set_ylabel('Santa Clara County %')\n",
    "    for i, (name, x, y, *_) in leaders.iterrows():\n",
    "        ax.text(x + 0.7, y - 0.2, initials(name))\n",
    "    return leaders\n",
    "\n",
    "\n",
    "leaders = make_leaders([ # Data as of Mar 24, 2025 (Name, Initials, SMC, SCC)\n",
    "    ('Megan Gardner',  99.94, 25.21),\n",
    "    ('Matthew Ring',   83.16,  2.48),\n",
    "    ('Peter Norvig',   78.80, 38.83),\n",
    "    ('Barry Mann',     78.27, 31.09), \n",
    "    ('Catherine Kircos', 54.47, 16.04),\n",
    "    ('Elliot Hoff',    52.89,  6.13),\n",
    "    ('Greogory P. Smith', 51.37, 23.27),\n",
    "    ('Brian Feinberg', 36.76, 48.22),\n",
    "    ('Chris Okeefe',   32.17, 48.30),\n",
    "    ('Jason Molenda',   7.60, 56.15),\n",
    "    ('Jim Brooks',      6.17, 53.51),\n",
    "    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Eddington Number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def Ed_number(rides, units) -> int:\n",
    "    \"\"\"Eddington number: The maximum integer e such that you have bicycled \n",
    "    a distance of at least e on at least e days.\"\"\"\n",
    "    distances = sorted(rides[units], reverse=True)\n",
    "    return max(e for e, d in enumerate(distances, 1) if d >= e)\n",
    "\n",
    "def Ed_gap(distances, target) -> int:\n",
    "    \"\"\"The number of rides needed to reach an Eddington number target.\"\"\"\n",
    "    return target - count(distances >= target)\n",
    "\n",
    "def Ed_gaps(rides, N=9) -> dict:\n",
    "    \"\"\"A table of gaps to Eddington numbers by year.\"\"\"\n",
    "    E_km, E_mi = Ed_number(rides, 'kms') + 1, Ed_number(rides, 'miles') + 1\n",
    "    data = [(E_km + d, Ed_gap(rides.kms,   E_km + d), E_mi + d, Ed_gap(rides.miles, E_mi + d))\n",
    "            for d in range(N)]\n",
    "    df = pd.DataFrame(data, columns=['kms', 'kms gap',  'miles',  'miles gap'])\n",
    "    return drop_index(df)\n",
    "\n",
    "def Ed_progress(rides, years=range(2024, 2013, -1)) -> pd.DataFrame:\n",
    "    \"\"\"A table of Eddington numbers by year, and a plot.\"\"\"\n",
    "    def Ed(year, unit): return Ed_number(rides[rides['year'] <= year], unit)\n",
    "    data  = [(y, Ed(y, 'kms'), Ed(y, 'miles')) for y in years]\n",
    "    df = pd.DataFrame(data, columns=['year', 'Ed_km', 'Ed_mi'])\n",
    "    return drop_index(df)\n",
    "\n",
    "def count_rides(rides, unit='kms', distance=100) -> int:\n",
    "    return count(rides[unit] > distance)\n",
    "\n",
    "count = sum"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.1"
  },
  "toc-autonumbering": true
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
